Skip to content

deps: migrate pnpm internals to v11 (1100.x)#10293

Open
zkochan wants to merge 31 commits intoteambit:masterfrom
zkochan:pnpm11
Open

deps: migrate pnpm internals to v11 (1100.x)#10293
zkochan wants to merge 31 commits intoteambit:masterfrom
zkochan:pnpm11

Conversation

@zkochan
Copy link
Copy Markdown
Member

@zkochan zkochan commented Apr 14, 2026

Summary

  • Rename 17 @pnpm/* deps in workspace.jsonc to v11's @pnpm/<domain>.<leaf> naming (e.g. @pnpm/core@pnpm/installing.deps-installer, @pnpm/config@pnpm/config.reader), bump in-monorepo packages to 1100.0.0.
  • Adapt to v11 breaking API changes: mutateModules now uses allowBuilds: Record<string, boolean | string> (plus dangerouslyAllowAllBuilds) in place of the removed neverBuiltDependencies / onlyBuiltDependencies / ignoredBuiltDependencies trio.
  • Config.rawConfigConfig.authConfig rename; read network settings (maxSockets, fetchRetries, etc.) from typed Config fields instead of rawConfig ini lookups.
  • Convert flat auth rawConfig to the new configByUri: Record<string, RegistryConfig> shape for CreateStoreControllerOptions; switch generateResolverAndFetcher to createResolver since both callers only use .resolve (avoids the now-required storeIndex on createClient).
  • Rename sortPackagessortProjects, createPkgGraphcreateProjectsGraph, createOrConnectStoreControllercreateStoreController; supply now-required globalVirtualStoreDir to getPeerDependencyIssues.

Test plan

  • npm run check-types passes
  • bit compile — 310/310 components compile
  • install new dependencies (using pnpm) e2e — 5/5 pass
  • install missing dependencies (when all packages exist) e2e — 2/2 pass (covers add/tag/export/import/reinstall)
  • skipping compilation on install e2e — 2/2 pass
  • Broader e2e suite
  • CI green

zkochan added 16 commits May 5, 2026 14:28
- Rename 17 @pnpm/* packages in workspace.jsonc to new domain-based names
  (e.g. @pnpm/core → @pnpm/installing.deps-installer, @pnpm/config →
  @pnpm/config.reader), bump in-monorepo packages to 1100.0.0.
- Update TS imports across scopes/components/e2e.
- mutateModules: replace removed neverBuiltDependencies/
  onlyBuiltDependencies/ignoredBuiltDependencies with the v11 allowBuilds
  map + dangerouslyAllowAllBuilds flag.
- Config.rawConfig → Config.authConfig; read network settings from typed
  Config fields instead of rawConfig ini lookups.
- Convert flat auth rawConfig to the new configByUri: Record<string,
  RegistryConfig> shape for CreateStoreControllerOptions; switch
  generateResolverAndFetcher to createResolver (avoids the now-required
  storeIndex on createClient).
- Rename sortPackages → sortProjects, createPkgGraph →
  createProjectsGraph, createOrConnectStoreController →
  createStoreController; supply globalVirtualStoreDir to
  getPeerDependencyIssues.
- Replace @pnpm/installing.modules-yaml imports in e2e tests with a local
  readModulesManifest that parses .modules.yaml directly. The pnpm v11
  package is ESM and fails with 'Cannot require() ES Module ... because
  it is not yet fully loaded' when Mocha loads multiple test files in
  parallel.
- Drop @pnpm/network.fetch from npm-ci-registry.ts and use the built-in
  global fetch with a local retry loop.
The pnpm v11 packages are ESM, and the `teambit.harmony/envs/core-aspect-env` Mocha
runner in the build capsule fails with "Cannot use import statement outside a module"
when the spec files transitively `require('@pnpm/deps.path' | '@pnpm/lockfile.fs' | '@pnpm/error')`.

Running `bit test scopes/dependencies/pnpm` locally still passes (10/10), so the
coverage isn't lost for local development — just disabled in CI until the capsule
test env supports require() of ESM.
Pnpm v11 (#11189) restricts .npmrc to auth + registry settings only;
hoist-pattern now lives in pnpm-workspace.yaml, so the legacy test
premise no longer applies.
pnpm v11 removed the neverBuiltDependencies install option entirely. Bit's
public API still exposes it, so translate it into the new allowBuilds map
with values set to false. Also ensures dangerouslyAllowAllScripts respects
the blocklist.
pnpm v11's createAllowBuildFunction short-circuits when
dangerouslyAllowAllBuilds is true and ignores allowBuilds entries entirely.
To keep the 'allow all except X' pattern working, skip the flag when a
deny list is present and emit allowBuilds with just the deny entries.
The package is ESM in pnpm v11; loading it through a top-level CJS require()
in the build capsule crashes with 'Unexpected token export'. Dynamic import
bypasses Node's require-ESM machinery and uses the proper ESM loader.
Using new Function('return import(p)') prevents bit's compiler from
rewriting dynamic import() back to require(), so @pnpm/releasing.commands
truly loads through Node's ESM loader rather than require-ESM.
…lude

Exclude @babel/plugin-transform-dynamic-import in both the root babel
config and the aspect env babel config so Babel preserves native
import() in compiled CJS output. This lets packer.ts call
import('@pnpm/releasing.commands') directly instead of going through
the new Function('return import(p)') escape hatch.
Now that babel preserves native import() in CJS output, the inlined
readModulesManifest in three e2e tests and the hand-rolled
fetchWithRetry in npm-ci-registry can be dropped in favor of
await import('@pnpm/installing.modules-yaml') / '@pnpm/network.fetch'
inside the call sites. Type-only imports of Modules stay static
since they're stripped at compile time.
The legacy link is only needed for external repos still importing
`@teambit/legacy`. Since v11 no longer guarantees the package is
installed locally (pnpm-lock no longer carries it), throwing on
require.resolve aborts every install/link in fresh CI workspaces.
Swallow the resolution failure so linking continues.
Three pre-existing dead statements (a `throw` after `return`/`throw` in
each file) only started failing once the post-install oxlint bump from
1.12 to 1.62 enabled stricter `no-unreachable` detection.
Dynamic import() in TypeScript sources is rewritten to a require()-wrapped
Promise by @babel/plugin-transform-modules-commonjs, regardless of the
preset-env exclude. Move the import() call into a sibling .cjs file that
the babel compiler skips (isFileSupported is false for .cjs), so the
native dynamic import survives compilation and pnpm's ESM-only package
loads correctly inside the build capsule.
zkochan added 11 commits May 5, 2026 16:06
….yaml

pnpm v11 (#11189) restricts .npmrc to auth + registry settings; behaviour
flags such as hoist-pattern moved to pnpm-workspace.yaml. Add a
pnpmWorkspaceConfig option to the scope helper and switch the test to
write hoist-pattern there, restoring the assertion that workspace-level
hoist-pattern propagates into the capsule install.
Pare back the comments added during the v11 work to keep only the
non-obvious "why" notes — drop what-comments, JSDoc/inline filler in
the auth-config helper, the duplicate ESM justification on packer.ts,
and the empty-catch placeholder in dependency-linker.ts.
The premise — workspace .npmrc options propagating into capsule installs —
no longer applies under pnpm v11, and bridging this through pnpm-workspace.yaml
isn't worth supporting. Drop the test and the pnpmWorkspaceConfig helper hook
added for the rewrite attempt.
…best-effort"

This reverts the relevant change from e6ca4a7 and the comment touch-up
in b5a5332. The original CI failure was diagnosed before later fixes
landed and was not reproduced afterwards, so swallowing the error here
masked symptoms without addressing the root cause.
…ompat"

This reverts the spec deletions from 9fa93e8. Node 22's native
require(esm) lets the capsule's mocha runner load the transitive
ESM-only @pnpm/* deps without any wrapping; bit test runs all
12 specs successfully.
…ipping on ESM)

Local bit test passes on Node 22 thanks to require(esm), but the
core-aspect-env mocha capsule on CI still throws "Cannot use import
statement outside a module" on the same Node 22.22 image. The spec's
top-level import of `./lockfile-deps-graph-converter` triggers the
chain unconditionally, so describe.skip does not help — drop the spec
again until the prod module's ESM-only @pnpm/deps.path and
@pnpm/lockfile.fs imports get pushed through a .cjs shim.

Keeps pnpm-error-to-bit-error.spec.ts which only references
@pnpm/error via a type-only import.
The capsule mocha-tester pins @babel/register@7.18.9 (2022), whose
require hook trips Node's require(esm) path on transitive
@pnpm/deps.path / @pnpm/lockfile.fs loads. Forcing ^7.28 via
workspace.jsonc overrides applies the rewritten compile/worker
implementation, which preserves the require(esm) flow for ignored
paths. Restore the spec so CI can verify; bit test passes locally
with the override (12/12).
@pnpm/config.reader@1101.2.0 exposes the helpers pnpm itself uses to
turn a flat npmrc-style auth dict into the configByUri map. Drop the
local port (authConfigToConfigByUri + decodeBasicAuth) and call the
upstream functions directly so the parsing stays in lockstep with
pnpm — including basicAuth decoding, SSL key/cert/file handling, and
default-registry creds keying.
The @babel/register@^7.28 override in workspace.jsonc only applies to
the bit workspace install; the env capsule that runs MochaTest installs
its own deps from @teambit/defender.mocha-tester (which pins
@babel/register@7.18.9), so the override never reached the failing
runner. Newer @babel/register also still trips on the transitive
require(esm) chain in CI.

Removing both the override and the lockfile spec, leaving the small
pnpm-error-to-bit-error.spec.ts which is unaffected. Restoring the
spec needs a fix in @teambit/defender.mocha-tester (or a capsule-level
override mechanism).
…tern

Move the @pnpm/deps.path and @pnpm/lockfile.fs imports behind an exported
async init() that loads them through a .cjs shim so the build capsule's
mocha runner doesn't trip on the transitive ESM require chain. Helpers
keep using module-level slots synchronously, so convertLockfileToGraph /
convertGraphToLockfile and all internal helpers stay sync in shape.

Callers (pnpm.package-manager.ts:calcDependenciesGraph and
dependenciesGraphToLockfile) await init() once. The spec adds a single
top-level before(init) plus a small before() in the simple-case describe
that needed to defer its describe-body-level convertLockfileToGraph call
until after init runs. bit test passes 12/12 locally.
@zkochan zkochan marked this pull request as ready for review May 6, 2026 08:20
Copilot AI review requested due to automatic review settings May 6, 2026 08:20
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates Bit’s internal pnpm integration to pnpm v11-era packages (1100.x), updating dependency names and adapting Bit’s pnpm/yarn package-manager glue code to pnpm’s breaking API and module-format (ESM) changes.

Changes:

  • Renames and bumps @pnpm/* dependencies in workspace.jsonc to pnpm v11 package naming and versions.
  • Updates pnpm/yarn integration code for pnpm v11 APIs (config/auth shape, resolver creation, store controller creation, peer issues options, project graph/sorting).
  • Adds CJS shims + adjusts Babel config to safely load ESM-only pnpm modules in test/build environments without breaking mocha/Babel require chains.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
workspace.jsonc Renames pnpm internal deps to v11 package names and bumps versions.
scopes/workspace/workspace/workspace-aspects-loader.ts Removes unreachable code in error handling path.
scopes/typescript/typescript/schema-extractor-context.ts Removes unreachable return after throwing.
scopes/scope/objects/models/dependencies-graph.ts Updates pnpm dep-path import to new package name.
scopes/pkg/pkg/packer.ts Switches pnpm pack command loading to ESM-safe dynamic loader shim.
scopes/pkg/pkg/load-pnpm-pack.cjs New CJS shim to import() ESM-only pnpm releasing commands for packing.
scopes/harmony/aspect/babel/babel-config.ts Adjusts preset-env to avoid transforming dynamic import().
scopes/dependencies/yarn/yarn.package-manager.ts Updates pnpm overrides parser import to v11 package.
scopes/dependencies/pnpm/read-config.ts Switches pnpm config reader import to v11 package.
scopes/dependencies/pnpm/pnpm.package-manager.ts Adapts network config reading + lockfile graph converter init + updated pnpm module imports.
scopes/dependencies/pnpm/pnpm-error-to-bit-error.spec.ts Avoids importing ESM-only @pnpm/error by stubbing a PnpmError-like object.
scopes/dependencies/pnpm/lynx.ts Migrates pnpm install/resolver/store controller APIs and script/build policy wiring to v11.
scopes/dependencies/pnpm/lockfile-deps-graph-converter.ts Adds async init + ESM loading shim for ESM-only pnpm modules; adjusts types.
scopes/dependencies/pnpm/lockfile-deps-graph-converter.spec.ts Ensures converter init runs before tests; minor structure tweaks.
scopes/dependencies/pnpm/load-pnpm-esm.cjs New CJS shim to import() ESM-only @pnpm/deps.path and @pnpm/lockfile.fs.
scopes/dependencies/pnpm/get-registries.ts Uses config.authConfig in place of rawConfig for credential extraction.
scopes/dependencies/pnpm/get-proxy-config.ts Updates config type import and no-proxy config source.
scopes/dependencies/dependency-resolver/package-manager.ts Updates pnpm peer issues type import to v11 package.
scopes/dependencies/dependency-resolver/dependency-resolver.main.runtime.ts Updates npm resolver import to v11 package.
scopes/component/modules/merge-helper/merge-files.ts Removes unreachable throw err after throwing a new error.
e2e/harmony/pnpm-default-hoisting.e2e.ts Uses ESM-safe dynamic import for modules manifest reading.
e2e/harmony/pkg-manager-config.e2e.ts Removes pnpm-specific capsule rc test section; keeps Yarn case.
e2e/harmony/dependency-resolver.e2e.ts Uses ESM-safe dynamic import for modules manifest reading.
components/legacy/e2e-helper/npm-ci-registry.ts Switches pnpm fetch import to dynamic ESM import (@pnpm/network.fetch).
babel.config.js Ensures Babel doesn’t transform dynamic import() (needed for ESM-only pnpm deps).

Comment on lines +33 to +41
export async function init(): Promise<void> {
if (loaded) return;
const { loadEsm } = require('./load-pnpm-esm.cjs') as {
loadEsm: () => Promise<{ dp: typeof Dp; getLockfileImporterId: typeof GetLockfileImporterId }>;
};
const m = await loadEsm();
dp = m.dp;
getLockfileImporterId = m.getLockfileImporterId;
loaded = true;
Comment on lines 68 to 70
after(() => {
npmCiRegistry.destroy();
});
zkochan added 2 commits May 6, 2026 10:32
- lockfile-deps-graph-converter: cache the in-flight load promise so
  concurrent init() calls share the same single ESM load instead of
  racing each other and producing transient undefined dp/getLockfileImporterId.
- pkg-manager-config.e2e: rename the suite to reflect that only the
  Yarn rc-file case remains (the pnpm hoist-pattern variant was dropped
  with v11 since pnpm-workspace.yaml is the supported config path now).
Copilot AI review requested due to automatic review settings May 7, 2026 12:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 3 comments.

Comment on lines 1 to 4
import semver from 'semver';
import type { LockfilePackageInfo } from '@pnpm/lockfile.types';
import * as dp from '@pnpm/dependency-path';
import * as dp from '@pnpm/deps.path';

Comment on lines 10 to 13
(supportNpmCiRegistryTesting ? describe : describe.skip)(
'package manager rc file is read from the workspace directory when installation is in a capsule',
'workspace .yarnrc.yml is read by Yarn when installation is in a capsule',
function () {
this.timeout(0);
import { buildDependentsTree } from '@pnpm/reviewing.dependencies-hierarchy';
import { renderDependentsTree } from '@pnpm/list';
import type { Modules } from '@pnpm/installing.modules-yaml';
import { readModulesManifest } from '@pnpm/installing.modules-yaml';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants